Web Survey Bibliography
Survey research has historically relied on a probabilistic model to underlie its sampling frame. With rare exception online research is non
‐probabilistic. Research without the safety net of a probabilistic frame raises all kinds of alarms. Challenges as to the reliability of online research has become a growing crescendo as the ‐probabilistic nature of online research has become evident. However, not all sampling frames must be probabilistic. Unfortunately, no such standard metrics exist to track reliability in online sampling. In fact, whether they are access panels or social networks there are no standardized means of balancing panels or even comparing them. To confound the situation the commercially used convenience panels are vastly different from each other (Gittelman and Trimarchi, CASRO Panel Conference, February 2009, paper available). These differences are so far reaching that those who elect to use these sample sources are not only without a safety net, they are at considerable professional risk. We have completed analysis of eighteen American panels and have found that respondent aging, frequency of professional responders, other satisficing behaviors as well as dramatic differences between sociologic, psychographic and buying behavior segmentations make for a cacophony of differences seemingly impossible to correct. ‐panel comparisons themselves are rare with data from a very few having been presented on any scale. ‐liners, invalids, inconsistencies, etc.] for which we have developed standard quantitative measures, and (2) a mechanism for developing a family of sampling standards based upon segmentation by key variables such as, but not limited to, media, purchasing and psychographics. It is the new availability of global data that allows us to present universal standards that help us meet necessary requirements that are our focus in this conference. In addition, to measuring performance, we believe that there are three key requirements for standard panel metrics including: (1) the ability to capture panel performance variations consistent with the differing needs of sample users, (i.e. a broadcasting company might wish to anchor its sampling frame to media segments); (2) The ability to create a data base that is retrospective in that new sample sources can be added to the database without repeating the analysis and (3) a focus on indices that are pragmatic in their measure (i.e. We always view buying behavior as the most pragmatic.)
non
In this study we will present the results of an extensive global study covering forty countries. Within each country panels will be compared using a 17 minute questionnaire, 400 completes per panel. We hope to present five or more providers per market. No such extensive comparison has been done on a global basis. In fact, inter
Preliminary data (24000 interviews) shows evolutionary trends in convenience panel development. Between panel differences appear more extreme in the United States than in other markets.
We are proposing two sets of practices: (1) using panel performance metrics [professionals, speeders, straight
In this talk, we propose to use segmentation analysis as a new metric that will allow us to anchor online data in a new non probabilistic sampling frame. It is the existence of global data that gives us a rare opportunity to experiment with this new methodology. Our goal is to use segmentation in each country to create a fingerprint that can be consistently maintained by blending panels. By minimizing the variability from the segments through optimization and panel combination we will establish a means for stabilizing online data irrespective of the panels and sourcing modes from which they draw their origin. We cannot stabilize online data unless we provide it with a reference point to anchor itself; the segments are that anchor. As the sourcing models continue to shift, panels will age and shift with them; we need a reliable anchor that rises above these problems. It is essential that we explore tools to measure these changes. Without a means of comparison we cannot expect to measure drift nor can we expect to have a platform for predicting the future. We do not profess to be on the road to a new probabilistic framework but rather a platform for comparison and continuity. We believe that there is a theoretical population online that can serve this purpose. Using the database we have gathered that includes respondents from over 160 global panels (64,000 interviews) distributed among 40 global markets we shall introduce new methods to build “perspective”.
Based on this we will use our segmentation models as a means of creating a “convenience” sampling frame by averaging segments into a “Grand Mean.” Using optimization models we will select convenience panels that best reflect the grand mean and the proportions by which they best fit together. We shall give evidence for the efficiency of these strategies.
Conference homepage (abstract)
Web survey bibliography (4086)
- The Effects of the Initial Mode of Contact on the Response Rate and Data Quality in an Internet-Based...; 2009; Wiseman, F.
- Doing Research in the Real World; 2009; Gray, D. E.
- Conducting Mobile Surveys: A Hands-on Introduction to an Innovative Research Mode; 2009; Pferdekämper, T., Melcher, T.
- Internet-based surveys and urban design education: A community outreach graduate project in Redding,...; 2009; del Rio, V., Levi, D.
- Exploration of secondary students’ creativity by integrating web-based technology into an innovative...; 2009; Jang, S.-J.
- An experimental mixed mode design on a general population survey ; 2009; Eva, G.
- Declining Working Phone Rates Impact Sample Efficiency; 2009; Piekarski, L.
- Using Non-Probability Samples for Confusion Surveys - Mall Intercepts and the Internet; 2009; Ericksen, E. P.
- Using Debit Cards for Incentive Payments: Experiences of a Weekly Survey Study; 2009; Gatny, H. H., Couper, M. P., Axinn, W., Barber, J. S.
- Characteristics of Cell Phone Only, Listed, and Unlisted Telephone Households; 2009; Tarnai, J., Schultz, R., Moore, D.
- Cell Phone-Only Households: A Good Target for Internet Surveys?; 2009; Bates, N.
- Nonsampling Error Research in Practice; 2009; Brick, J. M., Kalton, G.
- Total Survey Error: Past, Present, Future; 2009; Groves, R. M.
- Envisioning the Survey Interview of the Future ; 2009; Conrad, F. G., Schober, M. F.
- Metrics for panel contribution: a non probabilistic platform; 2009; Gittelmam, S. H., Trimarchi, E.
- Are telephone Surveys a dying bread. How declining response rates can be explained and resolved; 2009; Degen, M., Obermüller, A., Schielicke, A.-M.
- Relation between values and topic of a survey in internet panel research; 2009; Vis, C., Marchand, M.
- The potential of mobile research: Implications for the future and the role of industry standards; 2009; Nelson, Li.
- Factors Contributing to Participation in Web‐based Surveys among Italian University Graduates; 2009; Cimini, C., Girottu, C., Gasperoni, G.
- Integration of different data collection techniques using the propensity score; 2009; Camillo, F., Conti, V., Ghiselli, S.
- Mode effects in Switzerland: non‐response and measurement error on the European Social Survey; 2009; Roberts, C.
- The mixing of survey modes: application to Laon web and face‐to‐face household travel survey...; 2009; Bayart, C., Bonnel, P.
- Reason analysis: an ambitious alternative for mixed‐mode survey design; 2009; Jerabek, H.
- An innovative open source strategy for the development of electronic questionnaires for statistical...; 2009; Degortes, M., Landriscina, M., Murgia, M.
- Response rates in multi actor surveys; 2009; Pasteels, I., Ponnet, K., Mortelmans, D.
- Unit non‐response in panel surveys: empirical finding from an experiment; 2009; Haunberger, S.
- Do cash incentives helps with RDD studies? Examination of results from a national and a statewide survey...; 2009; Miller, Y., Barger, K., Hearn, D.
- Are people sharing their mobile phones? Selection probabilities in cellular telephone surveys; 2009; Fuchs, M., Busse, B.
- Accuracy of Estimates in Access Panel based Surveys; 2009; Enderle, T., Münnich, R., Bruch, C.
- New developments in survey methodology for official statistics; 2009; Bethlehem, J.
- Survey cooperation: response to initial and follow-up requests - Recent experiences from the recruitment...; 2009; Bartsch, S., Engel, U., Schnabel, C., Vehre, H.
- Methodological Research for Longitudinal Surveys; 2009; Lynn, P.
- Using Mobile Phones to Administer a Working Memory Updating Task in a Survey - Cognitive Performance...; 2009; Schmiedek, F., Riediger, M., Lindenberger, U., Wagner, G. G.
- Accessibility of individuals for mobile phone surveys; 2009; Gabler, S., Häder, S.
- Mixed Modes and Measurement Error: Comparing face-to-face, telephone and web modes ; 2009; Hope, S., Nicolaas, G., Jaeckle, A., Lynn, P., Nandi, A., Campanelli, P.
- The Difficult but Essential Challenge of Designing Mixed-Mode Surveys; 2009; Dillman, D. A.
- Overcoming the challenges of measuring self-reported digital media use: Using feedback to increase data...; 2009
- The Internet survey ; 2009; Getka-Wilczynska, E.
- Modelling online survey participation among Italian university graduates ; 2009; Cimini, C., Girotti, C., Gasperoni, G.
- Coverage rates of mobile telephones and the Internet in Italy ; 2009; Fabbris, L., Gorelli, S.
- Imperfect frames and new data collection techniques ; 2009; Biffignandi, S.
- Web based survey methods workshop; 2009; Weiss, M.
- Web Surveys; 2009; Bosnjak, M.
- Internet Surveys and the Demise of the Matrix; 2009; O’Neill, G., Freedman, S.
- Assesssment of Potential Bias in Telephone Survey Estimates Due to Noncontact and Noncoverage When Household...; 2009; Khare, M., Wouhib, A.
- Experimenting With Speech Interactive Voice Response, Touchtone Data Entry and the Web for the National...; 2009; Cantor, D., Brick, P. D., Tourangeau, R., Conrad, F. G.
- Avoiding the ‘Hang ups’ - Understanding Mobile Phone Self-completion Surveys ; 2009; Johnson, A. J.
- The Impact of Technology on Research Quality; 2009; Milla, P.
- Respondent Technology Preferences; 2009; Miller, C.
- Getting In Line with Online: Making Sense of Chat-based Focus Groups, Ethnogrophies, Blogs, and More; 2009; Longo, J.